HLT & NLP within the Arabic world: Arabic Language and local languages processing Status Updates and Prospects WORKSHOP DATE: SATURDAY 31 May 2008 Workshop chair

Automatic versus interactive analysis for the massive vowelization, tagging and lemmatization of Arabic Fathi Debili, Zied Ben Tahar, LLACAN, INALCO, CNRS, France and Emna Souissi, ESSTT, Tunisia How could we produce annotated texts massively with optimal efficiency, reproducibility and cost? Instead of correcting the output of the automatic analysis with dedicated tools, as suggested currently, we found it more advisable to use interactive tools for analysis, where manual editing is fed in real time into automatic analysis. We address the issue of evaluating these tools, along with their performance in terms of linguistic ergonomy, and propose a metric for calculating the cost of editing as a number of keystrokes and mouse clicks. By way of a simple protocol addressing Arabic vowelization, tagging and lemmatization, we discover that, surprisingly, the best interactive performance of a system is not always correlated to its best automatic performance. In other words, the most performing automatic linguistic behavior of a system does not always yield the best interactive behavior, when manual editing is involved. Prague Arabic Dependency Treebank: A Word on the Million Words Otakar Smrz, Viktor Bielicky, Iveta Kourilova, Jakub Kracmar, Jan Hajic, Petr Zemanek Institute of Formal and Applied Linguistics, Charles University in Prague, Czech Republic Prague Arabic Dependency Treebank (PADT) consists of refined multi-level linguistic annotations over the language of Modern Written Arabic. The kind of morphological and syntactic information comprised in PADT differs considerably from that of the Penn Arabic Treebank (PATB). This paper explores the possibility to merge both of these treebanks into a uniform resource that would exceed the existing ones in the level of linguistic detail, accuracy, and quantity. It overviews the character of PADT and its motivations, and reports on converting and enhancing the PATB data. The merged, rule-checked and revised annotations, which amount to over one million words, as well as the open-source computational tools developed in the project are considered for publication this year. Arabic Named Entity Recognition using Conditional Random Fields Yassine Benajiba and Paolo Rosso, Natural Language Engineering Lab. Departamento de Sistemas Informáticos y Computación, Universidad Politécnica de Valencia, Spain. The Named Entity Recognition (NER) task consists in determining and classifying proper names within an open-domain text. This Natural Language Processing task proved to be harder for languages with a complex morphology such as the Arabic language. NER was also proved to help Natural Language Processing tasks such as Machine Translation, Information Retrieval and Question Answering to obtain a higher performance. In our previous works we have presented the first and the second version of ANERsys: an Arabic Named Entity Recognition system, whose performance we have succeeded to improve by more than 10 points, from the first to the second version, by adopting a different architecture and using additional information such as Part-Of-Speech tags and Base Phrase Chunks. In this paper, we present a further attempt to enhance the accuracy of ANERsys by changing the probabilistic model from Maximum Entropy to Conditional Random Fields which helped to improve the results significantly. Can the building of corpus-based Arabic concordances with AraConc and DIINAR.1 tackle the issue of Arabic polyglossia? Joseph Dichy and Ramzi Abbès, Université Lumière-Lyon 2 and ICAR (CNRS-Lyon 2) Research and development in Arabic NLP has nothing to gain from an oversimplified view of the Arabic language, considering the high level of linguistic variation commonly observed in the Arabic language as a whole, including Arabic vernaculars (or ‘Dialects’). The real challenge is both to keep hold of the current sate of knowledge in the description of linguistic variation – complex as it may seem, it is in itself a simplified representation when compared to real-world language use –, and at the same time, to seek efficient formalized approaches that allow software applications to operate at a sufficient level of granularity. In this paper, we will try and show how a reasonable level of granularity and of subsequent feasibility can be reached through a balance between the complexity of Arabic corpus data and present-day tools that have been originally devised for Modern Standard Arabic (MSA). Section 1 introduces the complex system of the competence of communication in Arabic (Arabic polyglossia). Section 2 suggests a mapping of what could become, in the future, a real-world hyper-base of corpora in the Arabic language. Section 3 is an endeavour at contributing to the above challenge, through the use of tools initially devised for MSA. These are the DIINAR.1 knowledge database (DIctionnaire INformatisé de l’ARabe, version 1) and the last born of the analyzers based on it, the AraConc concordance software, which has been devised to extract concordances and frequency lists in Modern Standard Arabic. Two types of actual results are made available: statistical results and concordances. Examples of how these tools can meet linguistic variation in Arabic, and begin to evolve towards a new generation of tools are given.